The Internet remains a growing public network. Many companies rely on communication over the Internet using Internet Protocol (IP) to facilitate their business endeavors. However, public access also comes along with security risks. To address enhancement of security on the Internet, the Internet Engineering Task Force (IETF) proposed Internet Protocol Security (IPSec). IPSec is designed to provide authentication and encryption for communication over insecure networks, such as the Internet. However, once a packet is encrypted it cannot be compressed. Modems with built-in compression, such as V.42 for example, cannot compress an encrypted packet (due to the randomization of the data when it is encrypted), and thus throughput of such modems was slowed. Accordingly, the IETF proposed Internet Protocol Payload Compression (IPComp) to move compression up in the protocol stack, so that it can happen prior to encryption (instead of at the link level, below IP, as in modems).
IPComp allows systems to negotiate a type of compression for exchanging information prior to encryption. Unfortunately, implementations of IPComp require IPSec-capable computers, because IPComp negotiation is performed using the same negotiation protocol as IPSec, namely, Internet Key Exchange (IKE). Even though, IPComp relies on IKE, there is no reason that IPComp cannot be used independently of IPSec, without encrypting and/or authenticating communications. Unfortunately, in the Microsoft Windows Operating System, there is no Application Program Interface (API) for independently off-loading data for IPComp in operating systems (independently of IPSec, that is). Hereinafter, the term API is used to indicate an entire set of programs, definitions, protocols, subroutines, etc. in an interface, as well as indicate any particular program, definition, protocol, subroutine, etc. within an interface.
There is an API for offloading IPSec to an intelligent network interface (sometimes referred to as a “network interface card” or “NIC”). An intelligent NIC is used to do computationally intensive network stack operations rather than using the host's central processing unit (CPU). This frees up the CPU for other activities. For offloading IPSec, IPSec out-of-band data is created at an application level and passed down to a NIC for encryption on a packet-by-packet basis—the IP packet, including IPsec headers in their correct locations in the packet, is given to the intelligent NIC, along with an index (pointer) into the local “Security Association Database”, which contains connection-specific data, such as encryption keys and which encryption algorithm is in use for this connection.
At the high side, an amount of data handed down is equivalent to the largest physical packet size (measured in bytes) that a network interface can transmit, also known as the Maximum Transmission Unit (MTU). For example, the MTU for an Ethernet interface payload is 1,500 bytes, less overhead such as IP and TCP headers (typically 20 bytes each in the case of IPv4, or 40 bytes and 20 bytes for IPv6 and TCP, respectively), as well as any options in use. If Ethernet is used, packets of data may be handed down in blocks of about 1,500 bytes each. An additional 14 bytes and 4 bytes are appended to such a packet for an Ethernet header and trailer, respectively, and thus maximum packet size for an Ethernet packet is 1,518 bytes.
In IPSec, an Authentication Header (AH) and/or an Encapsulating Security Payload (ESP) header are optionally inserted in a packet, along with an ESP trailer—containing the Integrity Check Value (ICV)—if ESP-based Authentication has been negotiated for this connection. Additionally, if IPComp is in use, it will insert a Compression Header (CH) between the IPSec AH and/or ESP headers and the remainder of the packet. The addition of one or more of these headers results in adding more bytes to a packet. Continuing the above example for Ethernet, if payload handed down from an application level to a network interface level is 1,460 bytes, such a packet payload may have to be broken up or fragmented for transmission after the extra headers needed by IPSec, or IPSec and IPComp. However, the hope is that with IPComp, the packet payload will be reduced sufficiently to accommodate the additional headers and all of the original payload. Fragmentation should be avoided, if possible, because performance suffers, since fragmented packets will not have maximum payload usage.
An approach to address fragmentation is use of an API for “Large Send Offload” (also known as TCP Segmentation Offload) for the Transmission Control Protocol (TCP). There are Large Send API supports three component features (which can be used independently or together), namely, TCP Segmentation, TCP Checksum computation, and IP Checksum computation. For purposes of clarity, a Large Send API is used to refer to one or more APIs for initiating a Large Send. For Large Send offloads, a network driver is configured to inform a WinSock stack as to an MTU size. So for example, rather than 1,500 bytes for Ethernet, the network driver would indicate an MTU of 64 kilobytes (KB), or a large multiple of the actual packet payload capacity. In response to such configuration information, an application would thus send fewer, larger data blocks to the protocol stack—larger than can fit into the link's MTU.
Continuing the above example, data would be sent down to a NIC in blocks of approximately 64 KB. For a NIC with Large Send capacity, namely an intelligent NIC, a Network Driver Interface Specification (NDIS) layer provides an exemplary IP and TCP header and a pointer to the large block of data to a NIC driver. This driver divides up such data into path MTU-sized blocks, less any overhead, and sends out successive packets until the intelligent NIC has consumed the entire data block. Continuing the above example, if no options are used, overhead comprises TCP and IP headers totaling 40 bytes, so a 64 KB block of data would be divided as 64,000/1,460, resulting in 43 full packets and one “remainder packet”. If fragmentation had been necessary, each packet would have ended up as two fragments, for over 80 total packets. Thus, fewer packets are used, because more packets are fully loaded, such as for example approximately 1,460 bytes of data in each packet except perhaps the last packet, which includes however many bytes that remain after transmitting the rest of the data in the large block.
The initial large data block that is passed to the intelligent NIC includes a prototype TCP/IP header that will be used to build the header for each packet that is sent based on this data. Each Large Send packet will have a slightly different TCP and IP header provided by the NIC, derived from the prototype header, because for instance, TCP sequence numbers must be incremented by such MTU-sized blocks, e.g., by 1,460 for each packet, and the TCP checksum will be different for each packet, since it depends on the contents of the packet data. However the TCP Source and Destination ports will be the same in each derived packet. At the IP layer, the IP Identification field must be different for each unique packet that is sent, and the IP header checksum will be different for each packet as well, if only because the Identification field is different in each derived packet. Additionally, the calculation of the TCP checksum (which covers a 96-bit IP pseudo-header, the TCP header, and all TCP packet data) and the calculation of the IP header checksum (which covers the IP version 4 header but does not depend on packet data) are conventionally offloaded to the NIC driver. However, as noted above, each of the packets shares common information too, such as IP source address and IP destination address, among other common information as is known, for example the initial TTL.
However, APIs for Large Send do not provide support for IPComp. In fact, there is no API that allows an application to request that compression be offloaded to a lower layer entity, such as an NIC or similar component, for Large Send, i.e., there is no compression “on/off switch” for an application (independent of IPSec). Accordingly, it would be desirable and useful to provide IPComp in the context of Large Send offload capability, by enhancing the Large Send capability with the addition of a simultaneous attempt to negotiate compression, which if successful, would enable the Large Send data blocks to be transmitted using fewer packets.